Repository navigation
Fix from_format AttributeError for ordinal tokens - #1075
rahul05ranjan wants to merge 2 commits into
Conversation
from_format crashed with a raw AttributeError instead of a ValueError
for every ordinal token: 'Do' in locales without ordinal suffixes
(bg, de, ru, ...), because the Do pattern builder called .values() on
locale.get('custom.ordinal') which is None there; and Qo/Mo/DDDo/do/
e/eo/Wo/wo in every locale, whose None entries went to
locale.translation(None).values().
- Do now falls back to plain \d+ for locales without ordinals,
mirroring Locale.ordinalize(), so format/from_format round-trips
work in all locales.
- Mo, Qo and DDDo get real parse support (month/quarter/day-of-year
are all consumed by _check_parsed).
- The remaining None tokens (do, e, eo, Wo, wo) raise the regular
'Unsupported token' ValueError.
Fixes python-pendulum#1074
Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
| candidates = tuple( | ||
| rf"\d+{o}" for o in (locale.get("custom.ordinal") or {}).values() | ||
| ) | ||
| if not candidates: |
There was a problem hiding this comment.
Italian can have a nonempty ordinal map and still format some numbers without a suffix:
d = pendulum.datetime(1982, 8, 1)
s = d.format("Mo YYYY", locale="it") # "8 1982"
pendulum.from_format(s, "Mo YYYY", locale="it") # ValueErrorThe Italian map only has other (°), but 8, 11 and 80 use many, so Locale.ordinalize() falls back to plain digits. The Do case also fails on main, but the new Mo and DDDo support is affected too. Could the candidates account for a missing ordinal category?
I checked the base and this head with the Python sources loaded in memory; I did not run the full suite.
There was a problem hiding this comment.
Good catch — fixed in db42e68. _ordinal_candidates() now always appends a plain-digit fallback alongside the suffixed forms, so ordinal tokens (Do, Mo, Qo, DDDo) accept numbers from unmapped plural categories: from_format("8 1982", "Mo YYYY", locale="it") now parses. Added your repro as a regression test (it also asserts format("Mo YYYY", locale="it") == "8 1982"), and suffixed forms still parse (e.g. "3° ago 1982").
Local run of the full suite: 1880 passed, 3 skipped, 1 pre-existing failure (tests/test_main.py::test_instance_with_aware_datetime_any_tzinfo) that fails identically on the parent commit in this environment (pure-Python fallback, no rust build available here) — unrelated to this change.
Locales like Italian map only some plural categories (custom.ordinal has just 'other' -> '°'), so numbers in unmapped categories (8, 11, 80 use 'many') are formatted as plain digits by Locale.ordinalize(). from_format previously only accepted plain digits when the locale had no ordinal map at all, so parsing its own output raised ValueError. Always append a plain-digit fallback candidate alongside the suffixed forms so every ordinal token (Do, Mo, Qo, DDDo) accepts what format() emits. Add the Italian 'many' case from the review as a regression test. Co-authored-by: rahul05ranjan <11793455+rahul05ranjan@users.noreply.github.com>
Fixes #1074.
Summary
from_formatcrashed with a rawAttributeError: 'NoneType' object has no attribute 'values'for every ordinal token:from_format("7 Juni 1982", "Do MMMM YYYY", locale="de")(no ordinals in de)AttributeError1982-06-07from_format("2nd 1982", "Qo YYYY", locale="en")AttributeError(any locale)1982-04-01from_format("6th 1982", "Mo YYYY")AttributeError1982-06-01from_format("158th 1982", "DDDo YYYY")AttributeError1982-06-07from_format(..., "do/e/eo/Wo/wo ...")AttributeErrorValueError: Unsupported token: ...Meanwhile
format()happily emits all of these (format("Qo YYYY")→'2nd 1982'), so no ordinal token could ever be round-tripped.Root cause
src/pendulum/formatting/formatter.py:Doentry in_LOCALIZABLE_TOKENScallslocale.get("custom.ordinal").values()—Nonefor locales without ordinals (bg, de, ru, pl, uk, da, sv, ...).Locale.ordinalize()(theformat()side) already falls back to the plain number there, so the parse pattern must accept plain digits too.Qo/Mo/DDDo/do/e/eo/Wo/woareNonein_LOCALIZABLE_TOKENS;_replace_tokenspasses that tolocale.translation(None).values()and crashes before reaching its ownValueError("Unsupported token: ...")guard.Fix
_ordinal_candidates(locale): builds\d+<suffix>patterns fromcustom.ordinal, falling back to\d+for locales without ordinals — mirroringLocale.ordinalize(), soformat()/from_formatround-trip in every locale.Do,Mo,Qo,DDDonow parse to their fields (day/month/quarter/day_of_year), all of which_check_parsedalready consumes.do,e,eo,Wo,wonow raise the regularValueError("Unsupported token: X")instead ofAttributeError.Verification
tests/datetime/test_from_format.py(Dowith/without locale ordinals incl.de,Mo,Qo,DDDo, and the cleanValueErrorfor the unsupported tokens).tests/datetime/test_from_format.pyis byte-identical before/after (those 49 failures are thetime-machine-gated travel tests;time_machineis unavailable in this environment — diff of failing test names verified withgit stash).tests/formatting+tests/parsing: identical failure set before/after (all previously-passing tests still pass).